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Summary 

This Report describes two sets of subjective test measurements of error 
visibility on pictures derived from digital YUV video signals sampled in compliance 
with CCIR Recommendation 601. The first set of tests measured the subjective 
picture impairment resulting from random, single-bit errors in either the luminance 
(Y) or chrominance (U/V) channels. The second set of tests measured the impair- 
ment in chroma-key performance produced by errors in the U or V components. 

The results show that one random single bit error per second in the most 
significant hit (MSB) of the Y channel is just perceptible, and 10 per second in the 
MSB of the U/V channel. In the chroma-key tests, errors in the MSB of both U 
and V were just perceptible at 3 per second, and in addition, errors in the lower bit 
significances of the U channel were more visible than the V channel. 
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Introduction 



Direct effects of errors in Y and U/V signals 



When a digital signal is transmitted through 
a communications link, it may be subject to 
impairment caused by bit errors. These errors can 
occur either singly or grouped in bursts. Single bit 
errors can occur in a transmission link which is 
subject to noise or inter-symbol interference. 

This Report describes results of 
investigations into the subjective effects of single 
random bit errors in a digital YUV signal. The 
investigation was aimed at establishing criteria 
for the limits of acceptable link performance. 



In contrast, bursts of errors may result from 
effects such as tape-recording drop-out, timing 
jitter, electrical interference, de-scrambling 
extension or error correction failure.. Investigation 
of the subjective effects of such burst errors will 
be the subject of later work. 



The YUV video signal used in the 
investigation was sampled in compliance with 
CCIR Recommendation 601, Line and field 
synchronisation information was not sent via the 
error insertion system, so avoiding any 
equipment-dependent behaviour associated with 
any particular method of recovering this 
information from the link. Therefore, in practice, 
the results obtained may have to be downgraded if 
the picture synchronisation is affected by bit 
errors. 



2.1. Experimental details 

Fig. 1 is a block diagram of the equipment 
used. The digital YUV signal was obtained by 
sampling the red, green and blue video component 
signals at the line-locked frequency of 13.5 MHz, 
and quantising to 8 bits resolution per sample. 
These signals were then digitally matrixed to give 
Y, U and V, each sampled at 13.5 MHz. The U 
and V signals were then digitally filtered to half 
the bandwidth and sub-sampled to half the sample 
rate. For the Y signal, the codes 16 and 240 
represented black and peak white with a coding 
range of ± 112 levels. The equipment used in the 
tests was constructed before coding level standards 
were agreed and therefore the luminance coding 
range was very slightly non-standard. In the 
standard coding range for luminance, peak white is 
represented by level 235. This small difference is 
estimated to cause a reduction in the measured 
impairment corresponding at most to 0.1 grade 
which is small compared to the standard deviation. 

The YUV signal comprised two, 8-bit 
parallel data streams, one stream being the Y 
component and the other being the sample-inter- 
leaved U and V components. The error generator 
unit was able to selectively insert randomly timed, 
single-bit errors into any significance of bit in 
either data stream, and was able to generate up 
to lO'' error events per second. The digital YUV 
signals were then converted back to analogue RGB 
form using filters defined in Reference 1. The 
resulting signals were displayed using a high-quality 



B- 



syncs- 



Analogue 

R RGB 

to '^ 
G Digital 

B yuv 

U/V 
syncs. 

"^ syncs. 
OUT 



Digital 

Error 

Inserter 



Digital 
Y YUV „ 
to ^ 
Analogue q 

U/V RGB 

B 



VARIABLE 
DELAY 



*■ — R 

- — G 

" — B 

to 
monitor 



■ syncs. 



Fig. 1 
Schematic of arrangement 
for the subjective assess- 
ment of direct YUV 
errors. 
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colour monitor in standard viewing conditions'' . 
Errors in the luminance or colour difference signals 
could give rise to R,G and B values less than zero 
or greater than full saturation. Limiting was there- 
fore required to prevent the arithmetic of the 
YUV-to-RGB matrix overflowing, which could 
cause very visible error effects. 

2.2. Test procedure 

The EBU 'Boats' and 'Blackboard' pictures 
were shown alternately throughout each subjective 
test session. Each test condition was viewed for 
20 seconds followed by a 10-second rest interval 
during which the signals to the monitors were 
switched off, The pictures were graded according 
to the discrete values of the CCIR 5-point Impair- 
ment Scale given below: 

CCIR 5-point Impairment Scale: 

5 Imperceptible 

4 Perceptible but not annoying 

3 Slightly annoying 

2 Annoying 

1 Very annoying 

Each test condition presented a specific error rate 
at a particular bit significance in either the Y or 
U/V channel. The conditions were ordered 
randomly and programmed into the error generator 
to be 'replayed' for the tests. 48 test conditions 
were assessed at each test session, of which 30 
conditions occurred once only and the other 18 
consisted of 9 pairs of identical conditions to be 
used as cross-checks for judging consistency in 
each observer. 

A total of 24 observers took part in four 
separate test sessions lasting for twenty-four 
minutes each. The observers were seated at either 
4— or 6— times picture height from the monitor. 

2.3. Results 

Figs. 2a and 2b give the mean impair- 
ment grades for errors inserted in the Y and 
U/V channels respectively with results for the 
'Boats' and 'Blackboard' slides shown separately. 
The grade assessments were spread with an average 
standard deviation of about 0.6 of a grade. Much 
of this deviation is probably due to the impairment 
being assessed with integer values of 1—5; it is 
thought that the figure would have been reduced 
by the use of a continuous rather than a discrete 
grading scale. 

Of the 24 people who took part in the 



tests, three sets of results were rejected on the 
grounds of inconsistency. The remaining 21 sets of 
results were used which gives a standard error for 
the mean of about grade 0.15 for each result. 

These results show that error visibility 
decreases very rapidly with bit significance. Errors 
in the MSB (and to a lesser extent the MSB-1) of 
the Y component of the 'blackboard' picture 
tended to be particularly visible as white dots on 
the blackboard area. 

The results for grade 4.5 impairment (errors 
just perceptible by about half the observers) are 
summarised in Table 1 which gives figures for 
events/second and the corresponding 

bit-error-ratios. 

In order to be able to compare relative 
error visibilities in video signals which may have 
different bit rates, it is considered that 'error- 
events-per-second' is a more appropriate measure 
for describing the error rate than 'bit-error-ratio'. 
However, even this description of error rate is 
not ideal. Error visibility is a function of the 
energy in an error, so the visibility will depend not 
only on the error amplitude but also on the error 
width, which is related to the number of pixels per 
line. Therefore, the visibility of errors in a given 
bit significance and at a given number of 
error-events-per-second will still depend to some 
extent on the video data rate. 

As the data rate in these measurements for 
each of the Y and U/V channels is 108Mbit/s 
or approximately 10® bits per second, it follows 
that an error rate of N events per second in a 
channel is the same as a bit-error-ratio of about 
N/10® for that channel. 

The results are quoted in terms of error- 
events-per-second occurring in the Y (or U/V) 
data stream which includes data corresponding 
to the blanking intervals. The number of 
'displayed' error-events-per-second occurring 
during the active picture is approximately 0.75 of 
the numbers given. If then, for example, the 
blanking intervals have been removed for 
transmission, errors in the MSB of the Y signal 
will be just perceptible at 0.75 error-events-per- 
second rather than 1.0 error-events-per-second 
(but still at a bit-error-ratio of 10~^ ). 

For errors in the most significant bits at 
rates which are at the threshold of visibility, the 
decision as to whether the errors are just 
perceptible or not will depend on the time taken to 
view the picture. For these tests the viewing time 
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was 20 seconds. It is possible to make an estimate 
of how the assessment would change if the viewing 
time were varied. 



Let p^ = probability /second of an error 

occurring 
and p2 - probability of seeing an individual 

error 



Then p.| p^ = probability of seeing an error 
in any one second 

The probabihty of seeing an error in a time 
T is then given by p(T) = 1 — (1 — p^ p^ )^ 

For example, at the threshold of visibility 
errors in the MSB of the Y channel were just 
perceptible to half the viewers (i.e. p(T) = 0.5) 



Fig. 2 — Error visibility for errors in 
bits of various significance. 
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Error rates for grade 4.5 on the impairment scale 



Component Bit Significance 



Just Perceptible 
Events/second Bit-error-ratio 
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in a 20 second viewing time. This gives a value for 
p^Pj equal to 0.034. Thus, if the viewing time 
were increased to around 87 seconds, according 
to the above equation the errors would be 
perceptible to 95% of the observers giving a grade 
of 4.0 rather than 4.5. 

Errors in the less significant bits are only 
visible at comparatively high error rates. The 
impairment is then more like that from random 
noise and the assessment is not particularly 
dependent on the viewing time. 

3. Effects of errors on 'downstream' 
chroma-kev 

3.1. The effect of errors in chroma-keying 
signal 

The previous results show that the direct 
effects of errors in the Y component are more 
visible than errors in Che U/V components. 
However, if a signal containing U/V (chrominance) 
errors is processed by chroma-key equipment, 
the effects of errors may be more visible. 
Accordingly, measurements were made to assess 
the sensitivity of a typically instrumented chroma- 
key process to chrominance errors. 

In chroma-key, two pictures, a 'foreground' 
picture and a 'background' picture are combined. 
Basically the foreground picture is processed to 
obtain a 'key' signal which controls the switching 
or mixing of the two pictures. The foreground 
picture is suppressed relative to the background for 
a range of colours (usually around blue) in the 
foreground scene. Fig. 3 shows a typical chroma- 
key colour range. 



Chrominance (U/V) errors in the foreground 
signal will cause unwanted chroma-key switching 
if, as a consequence, the foreground colour crosses 
the chroma-key decision boundary. 

Luminance (Y) errors in the foreground 
signal should not effect chroma-key although this 
may depend on the adjustment of processing 
equipment. 

Errors in the background signal will not 
effect the switching key and will in any case, only 
be visible in areas of the output picture where the 
background signal is keyed in. 

3.2. Experimental details 

Fig. 4 is a block diagram of the equipment 
used. 

As for the previous tests, analogue RGB 
signals were converted to digital YUV into which 
errors were inserted. The error generator was 
reconfigured so that errors could be injected 
independently into either the U or V components. 
The digital YUV signal was then reconverted to 
analogue RGB which was used as the foreground 
input to an 'Ultimatte 4' chroma-key unit.'' A 
Philips RGB test-card generator was used for the 
background picture. The foreground picture was 
a critical chroma-key test slide containing a comb, 
a wig, a glass and some coloured spheres against 
a blue background. The background formed 
approximately 60% of the picture. Because of the 
time required to adjust the chroma-key set-up for 
each picture, it was not possible to carry out 
measurements for a range of picture combinations 
during the tests. However, with the picture 
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Fig. 3 — Chroma-key operation: 
diagram showing range of colours 
over which foreground suppression 
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combination chosen, the error events were more 
than usually visible, so the test results tend towards 
'worst case', 

In initial tests it was found that, with errors 
in the U component, errors in the MSB-2 tended to 
be more visible than errors in the MSB-1. This was 
because it so happened that the blue key-colour 
in the foreground picture had, for the most part, 
a particular binary value for the U component such 
that an error in the MSB-1 moved it to the right 
in U/V space (i.e. further into the keying area 
shown in Fig. 3) whereas an error in the MSB-2 
moved it to the left and out of the keying area, 
causing unwanted switching back to the 
foreground picture. 

These visibility effects could have been quite 
different if the blue key-colour of the foreground 
had had a U component with a different binary 
value. It was decided, therefore, to obtain worst- 
case error-visibility measurements for each 
significance of bit. The error insertion system for 
the U component was accordingly reconfigured so 
that instead of logically inverting a bit of a 
particular significance, an error of the same 
magnitude was arithmetically subtracted from the 
U value; i.e. the U/V coordinate was always moved 



to the left in Fig. 3 by the error event. 



This phenomenon in the error visibility was 
not evident for errors in the V component because 
the blue of the foreground had predominantly a 
near zero or small V component value. In this case 
as can be seen from Fig. 3, an error causing a 
vertical shift in the L'/V coordinate in either 
direction can take the coordinate outside the area 
of foreground suppression. Therefore, errors in the 
V component were inserted in the more 
conventional fashion by logical bit inversion. 

3.3. Results 

The results of this second set of subjective 
tests are shown in Figs. 5a and 5b for errors in 
the U and V components respectively. The error 
rates for which errors were just visible to about 
half the observers (i.e. grade 4,5) are given in Table 
2. 

The data-rate for each of the U and V 
components is 54 x 10^ bit/s. Each table entry 
may be divided by this value to obtain bit-error- 
ratios. 
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processing. 

a) Average impairment grade 
tor errors of various magni- 
tudes in the U component 

b) Average impairment grade 
for errors in various bit 
significances in the V 
component. 



The relative sensitivity of chroma-iiey to 
errors in the U and V components depends on the 
key colour and on the keying angle (0) of the area 
for foreground suppression (Fig. 3). The keying 
colour determines the orientation of the 
suppressed foreground area. With the blue keying 
colour (i.e. close to the U axis), the effect of 
errors in the V signal is increased as the keying 
angle is reduced. However, values for the keying 
angle were not ascertainable from the controls 



of the Uitimatte equipment and it was therefore 
not possible to assess the effect of changing this 
angle on error susceptibility. Comparison of 
Tables 1 and 2 shows that the process of 
chroma-key makes chrominance errors much more 
visible. The difference in visibility is particularly 
pronounced for errors in bits of lower significance 
in areas of the picture where they cause a switch 
from the background picture to the blue of the 
foreground picture. 



TABLE 2 



Error-events-per-second for grade 4.5 on the impairment scale 



Channel 


MSB 


MSB-1 


MSB-2 


MSB-3 


MSB-4 to MSB-7 


V 


3 


10 


30 


2x 10^ 


MO'' 


V 


2 


30 


3x10^ 


>10'' 


>10'* 
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4. Comparison of error visibility in digital PAL 
and YUV 

Comparisons can be made between the 
results for YUV components and earlier error 
measurements made with composite PAL signals 
sampled at three times colour sub-carrier frequency 
and quantised to 8-bit resolution'*-^ . The PAL 
results, however, were obtained by inserting 
random errors into a serial bit stream and are 
therefore not presented in the same manner, i.e. not 
giving the visibility of errors in the different bit 
significances. 

The PAL measurements showed that the 
error visibility was the same for the picture 
displayed either in colour or in monochrome. 
This is consistent with the YUV results which show 
that luminance errors arc significantly more visible 
than chrominance errors. 

Comparing the results for monochrome 
PAL and Y is complicated by the fact that the 
coding ranges are different. In the PAL case the 
luminance signal occupies only about 140 of the 
256 quantising levels available because of 
allowance for synchronising pulses and 
chrominance headroom. Consequently an error in 
the MSB corresponds to 91% of the full range 
luminance signal for a PAL signal but only 57% of 
the full range Y signal. 

Bearing these points in mind, it was found 
that for PAL signals sampled at about 13.3 MHz 
with 8-bit resolution, the errors were perceptible 
by half the observers (i.e. grade 4,5) for a bit-error- 
ratio of about 10~^. This corresponds to a 
threshold of perceptibility of 1.3 error-events-per- 
second at each significance of bit. If it is assumed 
that, hke YUV, errors in the MSB are far more 
visible than in any other bit significance, then this 
result can be regarded as entirely due to errors seen 
in the MSB. This value of 1.3 is very close to the 
value of 1 per second for the YUV luminance 
channel MSB given in Table 1. 

5. Conclusions 

The impairments caused by bit errors in 
YUV component signals coded in accordance 
with CCIR Recommendation 601 have been 
assessed in a series of subjective tests. The results 
summarised in Table 1 show that errors in the 
luminance channel were more visible than errors 
in the chrominance channel. Errors in the MSB 
of the luminance channel were just visible at an 
event rate of one per second and errors in the 
MSB of the chrominance channel were just visible 



at 10 error events per second. In addition, error 
visibihty decreased very rapidly as a function 
of decreasing bit significance. For example, errors 
in bits of significance MSB-2 or lower were not 
visible, even at error rates as high as 10'* events 
per second. 



These results are broadly comparable with 
those obtained with PAL coded signals. 



A second series of subjective tests measured 
the visibility of U or V errors with chroma-key 
processing. These tests, summarised in Table 2, 
showed that the processed picture was about three 
times more sensitive to such errors compared with 
the direct effects of chrominance errors. Errors 
in the output picture were just perceptible at only 
2 or 3 error events per second in the MSB of the U 
or V components. In addition, the effects of low 
bit-significance errors in the U channel could be 
seen. For example, for the MSB-3 of the U 
channel, errors were just perceptible at a rate of 
2x10'' error events per second. 



The general conclusion is that since the 
subjective impairment due to errors is increased 
by downstream processing such as chroma-key, 
these processes should set the standards for the 
acceptable error rates in YUV PCM transmission 
and recording systems. Further work is planned to 
investigate the effects of burst errors and of error 
concealment. 
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